Skip to content

HADOOP-19668. Add SubjectInheritingThread and update Daemon to restore pre JDK22 Subject behaviour in Threads#8061

Merged
steveloughran merged 1 commit intoapache:trunkfrom
stoty:HADOOP-19668
Nov 12, 2025
Merged

HADOOP-19668. Add SubjectInheritingThread and update Daemon to restore pre JDK22 Subject behaviour in Threads#8061
steveloughran merged 1 commit intoapache:trunkfrom
stoty:HADOOP-19668

Conversation

@stoty
Copy link
Contributor

@stoty stoty commented Nov 7, 2025

Description of PR

First part of HADOOP-19574.
Add SubjectInheritingThread and update Daemon, but only update the classes using Daemon

How was this patch tested?

Tested together with HADOOP-19670.
Ran the full test suite with JDK25, and confirmed that it does not cause regressions
(i.e. only tests which also fail with Java 17 fail with Java 25)

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@stoty stoty mentioned this pull request Nov 7, 2025
Closed
4 tasks
@stoty stoty changed the title HADOOP-19668 Add SubjectInheritingThread and update Daemon to restore pre JDK22 Subject behaviour in Threads HADOOP-19668. Add SubjectInheritingThread and update Daemon to restore pre JDK22 Subject behaviour in Threads Nov 7, 2025
pan3793
pan3793 approved these changes Nov 7, 2025
View reviewed changes
Copy link
Member

@pan3793 pan3793 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, cc @steveloughran @slfan1989

@hadoop-yetus
Copy link

hadoop-yetus commented Nov 7, 2025

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ trunk Compile Tests _
+0 🆗 mvndep 8m 30s Maven dependency ordering for branch
+1 💚 mvninstall 28m 43s trunk passed
+1 💚 compile 18m 7s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 compile 15m 44s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 checkstyle 2m 57s trunk passed
+1 💚 mvnsite 6m 30s trunk passed
+1 💚 javadoc 5m 23s trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javadoc 5m 8s trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌ spotbugs 1m 5s /branch-spotbugs-hadoop-common-project_hadoop-auth-warnings.html hadoop-common-project/hadoop-auth in trunk has 8 extant spotbugs warnings.
-1 ❌ spotbugs 3m 1s /branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html hadoop-common-project/hadoop-common in trunk has 448 extant spotbugs warnings.
-1 ❌ spotbugs 3m 38s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2801 extant spotbugs warnings.
-1 ❌ spotbugs 4m 0s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html hadoop-hdfs-project/hadoop-hdfs in trunk has 291 extant spotbugs warnings.
-1 ❌ spotbugs 1m 4s /branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-nfs-warnings.html hadoop-hdfs-project/hadoop-hdfs-nfs in trunk has 16 extant spotbugs warnings.
+1 💚 shadedclient 26m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 30s Maven dependency ordering for patch
+1 💚 mvninstall 4m 5s the patch passed
+1 💚 compile 14m 36s the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚 javac 14m 36s the patch passed
+1 💚 compile 16m 55s the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚 javac 16m 55s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 3m 18s /results-checkstyle-root.txt root: The patch generated 8 new + 189 unchanged - 0 fixed = 197 total (was 189)
+1 💚 mvnsite 6m 39s the patch passed
-1 ❌ javadoc 0m 47s /results-javadoc-javadoc-hadoop-common-project_hadoop-auth-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt hadoop-common-project_hadoop-auth-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 1 new + 71 unchanged - 0 fixed = 72 total (was 71)
-1 ❌ javadoc 0m 45s /results-javadoc-javadoc-hadoop-common-project_hadoop-auth-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt hadoop-common-project_hadoop-auth-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 1 new + 58 unchanged - 0 fixed = 59 total (was 58)
-1 ❌ spotbugs 3m 59s /new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html hadoop-hdfs-project/hadoop-hdfs-client generated 2 new + 2799 unchanged - 2 fixed = 2801 total (was 2801)
+1 💚 shadedclient 26m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 3m 40s hadoop-auth in the patch passed.
-1 ❌ unit 21m 59s /patch-unit-hadoop-common-project_hadoop-common.txt hadoop-common in the patch passed.
+1 💚 unit 2m 54s hadoop-hdfs-client in the patch passed.
-1 ❌ unit 213m 51s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 unit 3m 31s hadoop-hdfs-nfs in the patch passed.
+1 💚 asflicense 1m 14s The patch does not generate ASF License warnings.
481m 33s
Reason Tests
SpotBugs module:hadoop-hdfs-project/hadoop-hdfs-client
Unknown bug pattern AT_NONATOMIC_64BIT_PRIMITIVE in org.apache.hadoop.hdfs.DataStreamer.work() At DataStreamer.java:org.apache.hadoop.hdfs.DataStreamer.work() At DataStreamer.java:[line 816]
Naked notify in org.apache.hadoop.hdfs.DataStreamer$ResponseProcessor.work() At DataStreamer.java:At DataStreamer.java:[line 1300]
Failed junit tests hadoop.security.ssl.TestDelegatingSSLSocketFactory
hadoop.hdfs.tools.TestDFSAdmin
Subsystem Report/Notes
Docker ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8061/2/artifact/out/Dockerfile
GITHUB PR #8061
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux 697ad653f49d 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 064e97b
Default Java Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions /usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8061/2/testReport/
Max. process+thread count 3468 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8061/2/console
versions git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@pan3793
Copy link
Member

pan3793 commented Nov 11, 2025

@steveloughran @slfan1989

I think this has already been reviewed in #7892, can we move this forward quickly?

@stoty I think the only thing is to fix a few checkstyle issues reported by Yetus

@slfan1989
Copy link
Contributor

slfan1989 commented Nov 11, 2025

@steveloughran @slfan1989

I think this has already been reviewed in #7892, can we move this forward quickly?

@stoty I think the only thing is to fix a few checkstyle issues reported by Yetus

I’ve shared my feedback in #7892. I’m giving this PR a soft +1, but I think we should remain patient for now. Thanks again for everyone’s efforts.

@slfan1989
Copy link
Contributor

slfan1989 commented Nov 11, 2025

@stoty Could we also include the Javadoc updates and the new SpotBugs fixes? Thanks a lot!

@stoty
Copy link
Contributor Author

stoty commented Nov 11, 2025

I've checked the reported Javadoc errors, both are for files that I haven't touched.

@stoty
Copy link
Contributor Author

stoty commented Nov 11, 2025 •
edited
Loading

I've also checked the two new SpotBugs warnings.
Those are pre-existing, they are only reported as new because the method name has changes run() -> work().
If those are fixed on trunk, then these will also go away.
(-2 in the run() methods, +2 in the work() methods )

@steveloughran
Copy link
Contributor

steveloughran commented Nov 12, 2025

ok, going with the approval.

@steveloughran steveloughran merged commit e57306e into apache:trunk Nov 12, 2025
2 of 6 checks passed
@steveloughran
Copy link
Contributor

steveloughran commented Nov 12, 2025

@stoty ok, merged. please do the next stage

@stoty
Copy link
Contributor Author

stoty commented Nov 12, 2025

Thanks a lot @steveloughran .

@pan3793
Copy link
Member

pan3793 commented Nov 12, 2025

awesome, thanks @stoty and all!

@steveloughran
Copy link
Contributor

steveloughran commented Nov 12, 2025

ok, having merged it, I can see the javadocs are unhappy

 * subject restoration logic. SubjectInheritingThread provides a {@link work()}
 * method instead, which is wrapped and invoked by its own final {@link run()}

you need a # in front of the work() and run() calls.
can you just add this as a followup with the same JIRA ID? no need for a new one. thanks

@stoty
Copy link
Contributor Author

stoty commented Nov 13, 2025 •
edited
Loading

I have added the hashmarks, and added another missing commit to SubjectUtil in #8062 , @steveloughran .

However, I wasn't able to find the error you mentioned.
I ran javadoc:javadoc, and searched for Subject, but the only related error could find in the output was the SubjectUtil missing comment one.

Where/How did you find the javadoc errors/warnings you mentioned ?

@steveloughran
Copy link
Contributor

steveloughran commented Nov 13, 2025

my IDE is flagging it in SubjectInheritingThread; only thing I'd noticed

@steveloughran steveloughran mentioned this pull request Nov 17, 2025
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@slfan1989 slfan1989 slfan1989 approved these changes

+1 more reviewer

@pan3793 pan3793 pan3793 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

Common HDFS trunk

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@stoty @hadoop-yetus @pan3793 @slfan1989 @steveloughran